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Evolutionary game theory is an abstract and simple, but very powerful way to model evolutionary 
dynamics. Even complex biological phenomena can sometimes be abstracted to simple two-player games. 
But often, the interaction between several parties determines evolutionary success. Rather than pair-wise 
interactions, in this case we must take into account the interactions between many players, which are 
inherently more complicated than the usual two-player games, but can still yield simple results. In this 
manuscript we derive the composition of a many-player multiple strategy system in the mutation-selection 
equilibrium. This results in a simple expression which can be obtained by recursions using coalescence 
theory. This approach can be modified to suit a variety of contexts, e.g. to find the equilibrium frequencies 
of a finite number of alleles in a polymorphism or that of of different strategies in a social dilemma in a 
cultural context. 
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I. INTRODUCTION 

Game theory was originally developed in the field of eco- 
nomics to study strategic interactions amongst humans 
[9| [54]. The "agents" who play against each other have 
a set of "strategies" to choose from. The payoff which an 
agent gets depends on its own strategy and the strategy of 
the opponent. A player can decide which strategy to play 
against an opponent of a given strategy. 

In evolutionary game theory players are born with fixed 
strategies instead, [28] which are considered to be inherited 
traits. As usual, we assume a population game in which 
every player effectively plays against the average opponent. 
The success of a strategy depends on the number of players 
of that strategy and also the number of players with other 
strategies. A classical example is the Lotka-Volterra equa- 
tion [19] [27] [53] . If the number of wolves increases then the 
number of hares will decrease in turn leading to a decrease in 
the number of wolves. Evolutionary game dynamics studies 
the change in the frequencies of the strategies [31], which 
depends on mutation, selection and drift. 

A recurrent and obvious question asked in the study of 
games is which is the best strategy? Assuming an infinitely 
large population we can approach this question by the tradi- 
tional replicator dynamics [19]. The frequency of a strategy 
will increase if its average payoff is greater than the average 
payoff of the whole population. That is, if the individuals of 
a particular strategy are doing better on average than the 
individuals of other strategies then that strategy spreads. 
The average payoff of a strategy is also dependent on the 
frequency of the strategy. For finite populations one must 
resort to stochastic descriptions [8|[32]|41]. One important 
quantity is the fixation probability. Consider two strategies 
A and B in a population of size TV. Let the population be 
almost homogenous for B with only a single A. If there is 
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no fitness difference amongst the strategies, i.e. selection 
is neutral, then the probability that the A individual will 
take over the entire population is 1/N. If this probability 
is greater than 1/N we say that strategy A is favoured by 
selection. When there are multiple strategies in the pop- 
ulation, then a pair-wise comparison between the fixation 
probabilities of all the strategies will reveal which is the 
most abundant strategy [11] [THJ [16] [43] 151] . This analysis 
requires the assumption of low mutation rates. 

When mutations become more frequent then the concept 
of fixation itself is problematic and hence also that of fix- 
ation probability. In such a case we resort to the average 
frequency of a strategy which is maintained at the mutation- 
selection balance. This has been termed as the abundance 
of a strategy [3j. 

Consider n strategies which are effectively neutral against 
each other. In such a case the abundance of all the strate- 
gies in the stationary state will be just 1/n. Usually there 
are fitness differences between the strategies. If the abun- 
dance of a strategy is greater than that of all the other 
strategies then we can say it is favoured under the effects 
of mutation, selection and drift. Hence for n strategies, the 
k th strategy will be favoured if the abundance of k is greater 
than 1/n. Calculating the abundance of a strategy is a non- 
trivial exercise even when assuming weak selection. [3] have 
developed such an approach based on coalescence theory for 
the case of two-player games and n strategies. Under cer- 
tain conditions and weak selection, one can calculate the 
most abundant strategy for arbitrary mutation rates even in 
structured populations [2] [47] [48] and bimatrix games [35] . 

Usually two-player interactions are studied in evolution- 
ary game theory. The analysis of Antal et al. is also for 
two-player games. The interactions which we usually use as 
examples in evolutionary game theory are in general multi- 
player interactions making the systems nonlinear [33]. A 
classical example where a certain minimum number of in- 
dividuals are required to complete a task is group hunting. 
[45] studied cooperative hunting in lions. A typical hunting 
strategy is to approach the prey from at least three sides 
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to cutoff possible escape paths. This hunting approach is 
impossible with only two hunters, i.e. a two-player game the- 
oretic approach would be insufficient to capture the dynam- 
ics. Although evolutionary dynamics of multi-player games 
has received growing interest in the recent years, the main 
focus has been the Public Goods Game and its variants 
0[I31II3l23l3Sll3ZlHJaHllE2I. Only few authors consider 
general evolutionary many person games [121 HH EH] ■ We 
extend the approach developed by [3] for two-player games 
and multiple strategies to multi-player games. We show 
that in the limit of weak selection it is possible to calculate 
analytical results for n strategies and d players for arbitrary 
mutation rates. For a three-player game the mathematical 
analysis is described in detail. It is followed by an example 
with simulations supporting the analytical result. Lastly we 
discuss how the methodology can be extended for d-player 
games and argue that a general approach is possible, but 
tedious. 



II. ABUNDANCES IN THE STATIONARY STATE FOR 
THREE-PLAYER GAMES 

[3] have developed an approach to find the abundances 
of n strategies in a two-player game (d = 2). For a two- 
player game even with n strategies, the payoff values can 
be represented in the usual payoff matrix form. They can 
be represented as quantities with two indices, a^. We 
increase the complexity first by adding one more player (d = 
3). This adds another index for the third player's strategy 
set, ak,h,i- To calculate the average change in the frequency 
of a strategy we thus need to take into account this payoff 
'tensor'. 

We calculate the abundance of a strategy at the 
mutation-selection equilibrium. We begin by checking if 
there is a change in the frequency of a strategy, say k on 
average, due to selection. The average change under weak 
selection is given by 



N 



^ ^ ^k,h,i (%k%h%i) ^ ^ &h,i,j (%k%h%i%j) 



where the angular brackets denote the average in the neutral 
stationary state. The S (selection intensity) as a lower index 
on the left hand side, however, denotes that the average is 
obtained under (weak) selection. If we pick three individuals 
in the neutral stationary state, then the probability of the 
first one to have strategy k, the next one h and the last i, 
is given by the angular brackets in the first sum, (xkXhXi). 
Furthermore, a^h.i denotes the payoff values obtained by 
a strategy k player when pitted against two other players 
of strategy h and i. For n strategies the sums run from 1 
to n. This equation is the special case of a d — 3 player 
game. The derivation for arbitrary d is given in [A] The 
above equation is similar to the replicator equation, which 
is also based on the difference between the average payoff 
of a strategy and the average payoff of the population, but 



as we will see below, here the averages on the right hand 
side also include mutations. 

To incorporate mutations in the process, we write the 
total expected change due to mutation and selection as 
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The first term is the change in the frequency in the absence 
of mutation. In presence of mutations, the second term 
shows that the frequency can increase by l/(nN) by ran- 
dom mutation and decrease by x^/N due to random death. 
A mutation means that with a certain probability u, the 
strategy k can mutate to any of the n strategies. 

We are interested in the abundance of a strategy in the 
stationary state. In the stationary state, the average change 
in frequency is zero, (Ax i j? t }§ = 0, as the mutations are 
balanced by selection. Averaging Eq.|2]under weak selection 
thus gives us 
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This is our quantity of interest, the abundance of a strategy 
when the system has reached the stationary state. For d = 2 
player games, this quantity is given by [3j. For the abun- 
dance of a strategy to be greater than neutral, (xk)s > ^. 
the change in frequency in the stationary state due to se- 
lection must be greater than zero, (Ax s k el )§ > 0. 

Thus, we need to resolve the right hand side of Equation 
[T] Consider the first term in the brackets. In the neutral 
stationary state the number of combinations in the sums 
reduces due to symmetry, e.g. (xiXjXj) = {xjXiXj) = 
(xjXjXi). Hence, we need to calculate only three different 
terms, (xixixi), (X1X2X2} and (X1X2X3). Also for d player 
games, the terms in the sums are reduced. For the sec- 
ond term in the brackets we need to calculate five different 
types of averages, {x\X\XiX\) , (£1X2X2X2), (X1X1X2X2), 
l \X\X\X2X%) and (X1X2X3X4). These averages are derived 
in the [B] Using an approach from coalescence theory, we 
derive Si, the probability that i individuals chosen from the 
stationary state all have the same strategy. Hence 54 is 
the probability that four individuals chosen in the station- 
ary state all have the same strategy. If there are in all n 
^ 'strategies, then the probability that all have exactly strat- 
egy 1 is s±/n. Hence, (xiXiXiXi) = (^2X2X2X2) = ... = 
(x n x n x n x n ) = s^/n. Conversely, is the probability that 
if we choose i individuals in the stationary state, each has 
a unique strategy. Knowing these averages helps us resolve 
Eq. (0, 
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where \i = Nu and and are functions consisting 

only of the number of strategies n and the payoff values 
a>k,h,i (see[B]). Using this and evaluating Eq. ^ gives us 
the abundance of the k th strategy. 
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This expression is valid for large population sizes, NS <C 1 
and any \i = Nu. Since we have < u < 1, fi is bounded 
by < ii < N. 

We arrive at the result with an implicit assumption that 
there are at least four strategies. For n < d, each player 
cannot have a unique strategy and hence we need to set the 
corresponding terms to zero (see[B]). If there are less than 
n = 4 strategies then 54 vanishes. This does not affect 
our general result as the affected terms in L kl M k and H k 
simply vanish. 



III. AN EXAMPLE FOR THREE-PLAYER GAMES WITH 
THREE STRATEGIES 



Three-player game 
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Antal T, et al. J Theor Biol. 2009, 259 



Supplementary Material 



To illustrate the analytical approach we explore an evo- 
lutionary three-player game with three strategies A, B and 
C. Let our focal individual play strategy A. The other two 
players can play any of the three strategies. This can lead to 
a potential complication. Consider the combinations AAB 
or ABA. If the order of players does not matter, then both 
these configurations give the same payoffs but if they do 
matter then we need to consider them separately. Here we 
assume random matching, and hence the order drops out 
(e.g. AAB and ABA are equally likely). We consider an 
arbitrary game as denoted in Table [T] 

We need to calculate the average change in the frequency 
of strategy A due to selection, i.e. Eq. ([!]). We denote 
the co-efficients of the averages in the first sum by a±, 
a 2 and as- Hence for example, as — cla,b,c + &a,c,b- 
Similarly for the second sum we have ft to ft (Note that 
ft = oi\ = cla,a,a)- Thus we have, 
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and 
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FIG. 1. The average change in the frequency of strategy k 
due to selection, (Ax s k el )s for a three-player game. Notice first 
the similarity to the replicator equation where also we look 
at how a strategy is faring compared to the population. The 
first term in the bracket is analogous to the average fitness of 
strategy k. If we pick three individuals in the stationary state, 
then the probability that the first one has strategy /c, second 
h and the third i is given by (xkXhXi) (dashed box). Even for 
n strategies there are only three possible combinations, either 
all can have the same strategy, a pair has the same strategy 
or all three have different strategies. These probabilities were 
calculated by [3]. The s^'s appearing in the averages are the 
probabilities that if we choose i individuals from the station- 
ary distribution then they all have the same strategy. The 
second term in the bracket is analogous to the average fitness 
of the population in the stationary state. For this we need 
to pick four individuals and look for all the different combi- 
nations (solid box). For n strategies, five combinations can 
explain all the different configurations. These range from all 
the individuals having the same strategy (x \X\X\X\) to all 
having a different strategy (x 1X2X3X4) |b]). For the latter, we 
calculate Si , the probability that we choose i individuals from 
the stationary distribution and each of them has a unique 
strategy. For a general d-player game we need to pick d indi- 
viduals for the first term and d + 1 for the second. 



f3 4 (x A x A x B x c )- 
(7) 

Note that the term (x A x B xcx B ) which would appear with 
a factor /3 5 , does not appear, as we have only three strate- 
gies and thus 54 = 0. This also alters the definition of 
(x A x A x B xc) and (x A x A x B x B ) (see Figure [l] all terms 
dependent on s 4 are affected). 

We know the form of and from [B] as, 



This gives us the abundance of strategy A at the muta- 
tion selection equilibrium. Repeating the same procedure 
for strategies B and C gives the analytical lines in Figure[2] 
Although the analytical solutions are valid for large popula- 
tion sizes only, we still see a good agreement between the 
simulation and theory results, even for a population size as 
small as 30. 
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■ 3p 2 - 2p 3 ~ Pa] 
(9) 

H k = ± [n(ai + 0L2 + a s ) - (ft + (3 2 + ft + ft + ft)] (10) 

With L/e, Mk and H k as above, Eq. ^ for n = 3 reduces 
to, 
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IV. ABUNDANCES IN d > 3 PLAYER GAMES. 



We can repeat the whole procedure for d — 4 player 
games with n strategies. The formula for the abundance 
remains the same, Eq. ([3]), but the average change due to 
selection, Eq. ([!]), becomes more complicated. We need to 
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FIG. 2. For a three-player game with three strategies 
(d — 3; n — 3) we plot the average abundances of the three 
strategies as a function of the mutation probability u. The 
payoff table from Table [I] is used. The lines are the solu- 
tions of Eq. ([3]) and the symbols are the simulation results for 
the three strategies. Although the calculations are valid for 
large populations we see a good agreement even for a popula- 
tion size of N — 30 (selection intensity 5 — 0.003, simulation 
points are obtained averaging over 20000 independent runs, 
each over 2 x 10 6 time steps after a transient phase of N time 
steps). 
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TABLE I. An example payoff table for d = 3 and n — 3. 
Consider a three-player game with three strategies A, B and 
C. The strategy of the focal individual is in the column on 
the left. For example the payoff received by a C individual 
when playing in a configuration of CAB is 12. From the focal 
individual's point of view there are two ways of this config- 
uration CAB and CBA as it is twice as likely as compared 
to e.g. CAA. Hence we weight that payoff value by 2 when 
calculating the average payoff of strategy C. 



add an index in the sums, 

(Axf, el )§ = — ^ ^ ] &k,l,m,n{ x k x l x m x n) 
l,m,n 

^ ^ &l,m,n,o{ x k x l x m x n x o) S j (-^) 
l,m,n,o 

The first term is comparatively simple as we already know all 
the different ways of picking four individuals. For the second 
term we need to know the different possible combinations 
of strategies when picking five individuals from the neutral 
stationary state. 

For d players and n strategies we can construct an expres- 
sion analogous to Eq.[l] Consider for example the strategies 



played by d individuals denoted by, n, r2, r% . . . r^. Note 
that each of these can be a strategy from the strategy set 
l,2,3...n. Let k be our strategy of interest. Then the 
expression for the change of strategy k due to selection is 
given by, 




(13) 

where the sums range as usual from 1 to n ([A]). Solving this 
and plugging it in Eq. ^ gives the generalized expression 
for the abundance of strategy k for an n strategy, d-player 
game. We see that in the first sum the averages are for 
choosing d players but for the second it is d + 1. Hence 
we need to calculate the probabilities of the form Sd+i, but 
Sd+i depends on Sd- Thus we have to solve the expression 
recursively e.g. for d = 6, we will need to pick 7 players at 
most and we must solve the expression for d = 2,3,4,5,6 
before (d = 2 has been solved by [3] and d = 3 in this 
paper). As d increases calculating Sd+i is not enough and 
we will also need to calculate terms such as Sd+i which is 
already the case for d = 3. 

V. SPECIAL CASE: TWO STRATEGIES, n = 2 

Games with two strategies have been very well studied. 
In two-player games with two strategies, one strategy can 
replace another with a higher probability if the sum of its 
payoff values is greater than the sum of the payoff values of 
the other strategy. This is valid under small mutation rates 
for deterministic evolutionary dynamics [20] . The result also 
holds for for different dynamical regimes under specific lim- 
its of selection intensity and mutation rates [l][T0l|32]. Re- 
cently it has been shown that this result can be generalized 
for d-player games with two strategies [T2l [25]. 

Hence the condition which we find for d-player games 
should be identical to > derived in this paper for d 
players. We check this for d = 3, 

L k = \ [2ai(2 - 1) + 3a 2 - 2/3 2 - ft] (14) 
= §[201,1,1 + 3(a M , 2 + ai,2,i + ^1,2,2) 

— 2(ai ? i 5 2 + a l,2,l + a 2,l,l + a 2,2,2) — a l,2,2 ~ a 2,l,2 ~ a 2,2,l] 

(15) 

Thus L/c > is equivalent to, 

201,1,1+01,1,2 + Oi,2,l + 2oi, 2 ,2 

> 2o 2 ,l,l + 02,1,2 + ^2,2,1 + 2o 2 ,2,2 

(16) 

If we assume that the order of players does not matter then 
we have 01,1,2 = a i,2,i and 02,1,2 = ^2,2,1- This yields 

«1,1,1 + «1,1,2 + ai,2,2 > 02,1,1 + 02,1,2 + 02,2,2, (17) 
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TABLE II. Payoff table for a two-player game with three 
strategies, T± i.e. specialising in task 1 or T2 i.e. specialising 
in task 2. Ti produces enzyme 1 and T2 produces enzyme 
2. When both the enzymes are present then a benefit b is 
obtained. The cost of producing enzyme 1 is c\ and the cost 
to produce enzyme 2 is C2. If the partner in the game is of 
the same strategy then the cost is shared. The freeloading 
strategy F does not pay any cost and so the benefits of the 
resource are unavailable to it. 
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TABLE III. Payoff table for the same game as discussed in 
Table [il] T\ and T2 refer to specialising in task 1 and 2 namely 
producing enzyme 1 and 2. Note the costs can also be shared 
if at least one of the game partners is of the same strategy. 
In this case the freeloaders can get the benefit when the other 
two players produce both enzymes. 



which is exactly the condition that has been obtained previ- 
ously using different methods and different notation by [25] 
and [12]. 



certain range of mutation probability. 



VII. DISCUSSION 



VI. APPLICATION TO A TASK ALLOCATION 
PROBLEM 

To demonstrate the power of the approach we are mo- 
tivated by [55] who studies the problem of task allocation 
and of the evolution of division of labour. [55] studied a 
two-player game between task 1 specialists (Ti), task 2 
specialists (X2) and generalists. Instead, we have T\, X2 
and freeloaders F. We can think of this problem in con- 
text of bacteria that need two types of enzymes to obtain 
resources from the environment. One strain produces one 
type of enzyme at a cost c\ and another strain produces the 
second type of enzyme at a cost of C2. We also have the 
freeloading strain which does not produce any enzyme but 
can get resources by the help of the other two strains. The 
benefit of getting the resources is given by b. We have the 
condition that the total cost is less than the benefit accrued 
i.e. b > ci + C2. Further assume that our contenders are 
conservative in the enzyme production. Sensing who they 
are pitted against, the strains share the costs of producing 
the enzyme. Thus a two-player payoff matrix for such a set- 
ting can be written down as in TablefTT] It is hard to imagine 
though that the bacteria interact only in a pair-wise fashion. 
Although it is hard to judge how many players are interact- 
ing, we can at least increase the complexity by one more 
player and study what effect this has on the abundances of 
the strains. Therefore, we study the system in a three-player 
setting. In this case the payoff table will look like [NT] 

We calculate the abundances of the strains in these dif- 
ferent settings, cf. Figure[3j Even when there are almost no 
mutations there is a quantitative difference between the av- 
erage frequencies of the strains. For a higher mutation prob- 
ability the difference is also qualitative. While the freeload- 
ers never pay a cost they have the highest abundance in 
the two-player setting for any mutation probability. For the 
same reasoning but in three-player games we see that the 
abundance of the freeloaders falls below that of X2 for a 



Public goods games are often used as examples of multi- 
player games. In the beginning there were the cooperators 
and defectors. Then came the punishers and then the loners 
[13] [46]. Now we talk about second order punishers, pool 
and peer punishers (43] and more. Studying these systems 
for small mutation rates and arbitrary selection intensity 
is almost becoming standard [11] [15] HU [43l [51] . In the 
limit of weak selection our method allows to find out which 
strategy is most abundant for arbitrary mutation rates. 

Yet, another important aspect of most social dilemmas 
and many other biological examples is that they involve 
multiple players [5] [26] [30] [45]. [2] [3] have made use of 
the coalescence approach to characterize the mutation pro- 
cess under neutrality and then apply it under weak selection 
to two-player games with n strategies (n x n). Here we 
extend the approach to d-player games with n strategies. 

We give an example for an n x n x n game and derive 
the analogous expressions for abundances of the strategies 
for arbitrary mutation rates. When we increase the number 
of players to d, the payoff matrix becomes a d dimensional 
object. We run into the problem of whether the order of 
players matters or not. Either way this does not influence 
our results but notation-wise it is easier if the order of play- 
ers does not matter. Adding a new player adds a new index 
to the payoff values. For calculating the abundance we need 
to assess Eq. (13). For solving the two sums in Eq. (13) we 
need to know the different combinations of choosing d play- 
ers and d+ 1 players from the neutral coalescent stationary 
state. 

To illustrate the complexity of the situation take for ex- 
ample 54. This is the probability that four chosen individuals 
have the same strategy. ln[C]we have shown that deriving s 4 
depends on S3 which depends on 82 in turn. Hence in gen- 
eral to derive Sd+i, we need to know sa, s^-i, Sd-2 • • • , $2- 
In addition, for general d-player games we need quantities 
such as Sd+i, probability that d+\ individuals chosen in the 
stationary state all have different strategies. In the absence 
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FIG. 3. The strains of a bacteria T\ and T2 produce the enzymes 1 and 2 which when present together provide a benefit b. 
When more than one individual of the same strain is present then the production costs for the enzymes, a and C2 are shared. 
A third strain F does not produce any enzyme and thus avoids the costs and cannot obtain the benefit on its own. W e can 
analyse the interactions by assuming them to be pair- wise (two-player game, Table [Ilf or in triplets (three-player game, lllll) . 
We notice that the abundances of the three strains are qualitatively and quantitatively different in the two settings even though 
the underlying rules defining the interactions are the same. Under neutrality the abundance of all the strains would be given 
by the dashed line. The full lines are the analytical results obtained by solving Eq. ([3]), assuming b — 1.0, c\ — 0.6, C2 = 0.2 
and a population size of N = 30 with selection intensity 5 = 0.003. 



of mutations such probabilities as vanish and we are 
left with only Sd+i, as mentioned in (34]. If n < d then 
Sd+i is zero and hence the terms dependent on it need to 
be recalculated. After recalculation the terms which are af- 
fected either vanish or are automatically adjusted such that 
the result can be written again in the form for L^s, Mk's 
and Hfr's. However, for a d player game, d — 2 intermediate 
terms such as appear. 

For two strategies, Lj~ reduces to the general con- 
dition derived in [12J [25] and again holds for ar- 
bitrary d. For Hjz for any number of strategies 
we conjecture that it will always be of the form 

n (j2r 2 ,-r d a k,r 2 ,...r d ^ ~ (Z)r 2 ,...r d +i a r 2 ,...r d+1 ) 

from Eq. ( [l3| ). Addressing a generalization for d-player 
games is not a fundamental problem of the approach but 
requires a tedious recursive effort and poses a notational 
challenge. At the coalescence level the problem rests on 
permutations and combinations. 

Arbitrary mutation rates can be interpreted in different 
ways. In the social learning context [49] it can be thought 
of as the exploration rate where the players experiment 
with different strategies. A recent behavioral experiment 
suggests that humans change their strategies with a prob- 
ability much higher than usually considered in theoretical 
models (50]. Analytical results for such high mutation rates 
can now be provided utilising the approach outlined herein. 
Small mutation rates are most relevant in population genetic 
contexts where the strategies can be thought of as alleles. 
While most people think of evolutionary game theory as a 
phenotypic approach, one can as well consider evolution- 
ary games on the level of genes [6] fT7HT9l l38l l39l [42] . A 
long standing problem in theoretical biology is to explain 
the maintenance of polymorphisms. So far, the correspond- 



ing models typically assumed small mutation rates, but our 
approach provides a way to calculate the abundances for 
arbitrary mutation rates. However, our aim is not to cal- 
culate abundances, but to depict a way to derive them. A 
two-player interaction can capture the effect of one allele 
on another, but the epistatic interaction of many loci calls 
for a theory of many-player interactions. 

Making use of our approach we can precisely determine 
the composition of a population with any finite number of 
different types under weak selection for arbitrary mutation 
rates. Another convenient way of finding the strategy which 
performs the best is the limit of small mutation rates. For 
small mutation rates, the system spends most of its time in 
a monomorphic state. We can approximate the system by 
just looking at fixation probabilities of the different types. 
Our approach illustrates that the interaction of d players 
is significantly more complex than the usual two-player 
games. General multi-player games pose exciting challenges 
way beyond the usual intricacies of public goods games. 
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Appendix A: Deriving the average change due to selection 

We begin the Appendix by first deriving the average 
change in the frequency of a strategy under selection for 
an arbitrary number of players (d) and strategies (n). In 
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[B| we consider the special case d = 3. The more technical 
calculations based on coalescence theory can be found in 
Section \Q 

Let us begin with the simple case of a two-player game. 
The payoff matrix for a two-player game, A with n strate- 
gies is an n x n matrix, 



A = 





1 


2 


n 


1 


/ 01,1 




01, n 


2 


02,1 


^2,2 • 


• 02, n 


n 


\ ^71,1 


0n,2 • 


0n,n 



(Al) 



The average fitness of strategy 1 can be written down as, 



fi 




(A2) 



where the leading 1 is the baseline fitness and S > is the 
intensity of selection. The variable Xh is the frequency of 
players with strategy h, J2h=i Xh ~ 1- We assume that the 
5 is so small that the fitness is always positive. Similarly, 
for a three-player game the payoffs have an additional index. 
Thus we can write the average fitness of strategy 1 as, 



fi 



5^01,/M (x h Xi) 
h,i 



(A3) 



As usual, the sums run from 1 to n, the number of strate- 
gies. Continuing up to d players we now consider r 2 . . . rd 
the strategies of players 2 ... d as the strategy of one of the 
players is set to k, i.e. r\ — k. We see that the average 
payoff of strategy k can be written as, 



f fc = i + a( Yl 

\r 2 ,...,r ( 



0/c 



' _• r d (x r2 x r 3 ' ' ' x r d ) J (A4) 



The average payoff of the whole population is given by F 
as, 



(A5) 



k=l 



Now we need to consider the dynamics of the process. The 
Moran process is used, where in each time-step an individ- 
ual is chosen proportional to its fitness to reproduce and a 
randomly chosen individual dies. With probability 1 — uthe 
individual chosen for reproduction produces an exact copy 
as itself, but with probability u, a mutation occurs and the 
offspring can be of any of the n strategies. 

If the abundance of a strategy is greater than 1/n, then it 
is favoured at the mutation-selection equilibrium. To calcu- 
late the abundance of strategy k we begin with the average 
number of offsprings of an individual of strategy k, which is 
given by, 



1 



1 

TV 



NY' 



(A6) 



The first term captures the survival of the parent. The 
second and third terms refer to the random death and fitness 
proportional reproduction. For S <C 1, we have, 



S_ 
N 

E 



^ ^ 0/c,r2,...,rd (x r2 X r3 • • • X rd ) 
\r 2 ,...,r d j 



E 



x r d+1 / J a r 1: r 2 ,...,r d 



(x r2 X r3 ■ • • X rd ) 



(A7) 

The change in the frequency of strategy k, Xk, due to se- 
lection is given by, 

Ax s k el = x k uo k -x k . (A8) 

The vector x = (xi, . . . , x n ) contains all possible frequency 
compositions of the system. The system will be in state x 
with probability Q$(x.). Hence by averaging Ax s k el in the 
leading order of S we obtain, 




E a 



Q j k,r 2 ,...r d {x r2 X r3 . . . X rd ) 
r 2 ....r d+1 (%r 2 x r 3 • • - x r d+1 ) 



ir 2 ,...r d+1 




Thus we reach the expression for the average change in the 
frequency of strategy k due to selection in the stationary 
state as, 




r 2 ,...r d+1 



(A10) 

Notice the form of a replicator like equation in the above 
terms. We look for the difference between the average pay- 
off of a strategy and the average payoff of the population. 
The first sum consists of a product of d frequencies while 
the second sum requires a product of d+1. Particularly, we 
consider the case d = 3. For strategy k the average change 
due to selection is given by 

(Ax s k el ) 6 = I ^2a k ^i(x k x h Xi) - ^2a hiii j{x k x h XiXj) ] . 



h,i 



(All) 
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Appendix B: Three player games 
1. Choosing a set of players 



To solve Eq. ( Allty we need to solve the two sums on the 
right hand side. The first sum can be solved using the tech- 
nique derived by [3j. For the second sum we need to know 
the different forms of the averages possible. Using symme- 
try arguments such as (x 1X2X2X3) = (xiX2XsXs) (this is 
valid because we average under neutrality) only five differ- 
ent kinds of averages are required, (xiXiXiXi), (xi^^a^), 
(X1X1X2X2), {X1X1X2X3) and (X1X2X3X4). The quantities 
(xixixixi) and (X1X2X3X4) are derived in section [C] based 
on coalescence theory. The rest of the averages can be 
written down as, 



where the probability that if we choose i individuals from 
the stationary state of a neutral coalescent then all i have 
the same strategy is s^. The quantities 52 and 53 have 
been previously derived in [3]. For completeness we repeat 
the derivation in Section [C] In Section [C] 54 is calculated, 
which is the probability of choosing four individuals from the 
neutral stationary state and all have the same strategy. If 
there are n strategies then the probability that all four have 
strategy 1 is 54/71. Thus (x\X\X\X\) = s^/n. Similarly, the 
probability that all four have different strategies is S4. The 
exact case when the first individual has strategy 1 , second 
has 2 , third has 3 and the fourth has 4 is just (X1X2X3X4) = 
54/ (n(n — l)(n — 2)(n — 3)). Using this information we can 
get the expression for all the five averages as, 



(i) Three of a kind, (X1X2X2X2) 
(xix 2 X2X 2 ) = 



Xi X2X2X2 



U'-Sv---/ 

(xiXiXi) — (xiX\X\Xi) — (n — e i) l \X\X2X2X2) 
(xiXlffl) - (XxXxXxXx) 

" — 1 



(ii) Two pairs, {x\X\X2X2). 

(X 1 X 1 X 2 X 2 ) = ^(1 - X 2 - y^^X i )x 1 X2X2^l 

= (X1X2X2} - {XXX2X2X2) - (n - 2)(xixix 2 x 3 ) 

(B2) 

(iii) Single pair, (xiXiX2Xs). 

(xixix 2 xs) = ^ ^1 - x 2 - x 3 - ^ x-^j XiX 2 Xs^ 

= (xix 2 xs) - 2{x 1 xix 2 xz) — (n - 3)^X2X3X4) 
(X1X2X3} - (n — 3)(xiX2X 3 x 4 ) 



(xiX\XiXi) 

(xix 2 X2X 2 ) 

(XiXiX 2 X 2 ) 

(x 1 xix 2 x 3 ) 
(X1X2X3X4} 



S4 

n 
S3 



- S4 
n(n — 1) 

54 + 3^4 — 883 + 6S2 — 1 



1 



3n(n - 
3s 2 + 2s 3 



1) 

- s A 



3n(n - 



l)(n 

Si 



2) 



n(n- l)(n-2)(n-3)' 



(B6) 



The quantities S2, S3, 54 and 84 are derived in Section [C] 
Substituting these values above yields, 





= n(n + /i)(2n - 


f /i)(3- 


h/i)C 


(x 1 x 2 x 2 ) 


= nfi(n + /x)(3 - 


f /i)C 




(x 1 X2X 3 ) 


= n/i 2 (3 + /i)C 






(x\XiXiXi) 


= (n + /i)(2n + 


/i)(3n H 




(X1X2X2X2) 


= /i(n + /i)(2n - 


f fiC 




(xiXiX 2 X 2 ) 


= /i(n + /i) 2 C 






(X1X1X2X3) 


/i 2 (n + /i)C 






(X1X2X3X4) 


= /i 3 C. 







(B7) 



(B3) whereC= [n 4 (l + /i)(2 + /i)(3 + /i)]" 



2. Number of strategies with respect to the number of 
players 



Thus we can write all averages in terms of (x\X\X\x\) , 
(X1X2X3X4} and the known quantities from [3], 

(X 1 X2X2X2) — 

n 1 Now that we know the form of the averages, we can begin 

(x 1 x 1 x 2 x 2 ) = (x 1 x 2 x 2 ) - (x 1 x 2 X2X 2 ) - (n - 2)(x 1 x 1 x 2 x 3 ) expanding the sums from Eq. flAlip , first for d = 3 and for 

n > 3. Consider the first sum, 



(x 1 x 1 x 2 x 3 ) = 



(X1X2X3} - (n — 3)(xix 2 xsx 4: } 



(B4) 



From [3j we know the form of, 

S3 
n 

S2 ~ s 3 



(XiXiXi) 

(XIX2X2} 
(x 1 x 2 x 3 ) 



^2cik,h,i(xkXhXi) = (xixixi)a ki k,k 

h,i 

+ (XIX2X 2 ) ^ ak ^i 



n(n — 1) 

1 - 3s 2 + 2s 3 
n(n — l)(n — 2) 



h,i 

h=k,i^k 
i=k,h^k 



(B5) 



-(xiX 2 X 3 ) ^ a k,h,i- ( B8 ) 



h,i 
k^h^i^k 



For the ease of notation we denote the co-efficients on the Denoting the coefficients of fin 3 , a 2 n 3 and /i 3 n 3 as Mk 

right hand side by ql\ = ak y k,k, &2 = h,i ak,h,i, and Hk respectively, we get the following result 

k^h—i^k 

i^iit (Axl* l h = S A^±MhR±J^l (B 14) 

a 3 = E h,i a k , hti . Hence, we have, Nn(l + fi)(2 + fi)(3 + fi) 

k^h^z^k Next we consider d = 3 and n = 3. In this case the sums 
y^ak,h,i(xkXhXj) = (xixxxijax + {x x x 2 x 2 )a 2 + (xiX^W 1 Ec l- d A11 D are - 

h,i 



Similarly, the second sum in Eq. ( |A11| ) becomes, 

^2 ^h,i,j{x k X h XiXj) = (x 1 x 1 x 1 x 1 )f3 1 + (x 1 X 2 X2X 2 )f32 

J r{x 1 x 1 x 2 x 2 )j3 ?> + (xixix^s)^ 

+ (xiX2X 3 X4)/3 5 (BIO) 

Note that 0i = a\. Substituting the expressions for the 
averages from Eqs. flB7ty , 

y*lh i ^k,h,i{^kXhXi) 

c 

= n(n + a)(2n + /i)(3 + + na(n + /i)(3 + /i)a 2 

+n/i 2 (3 + /i)«3 
= 6n 3 ai + n \2n 2 a\ + 3n(3ai + a 2 )] M 

-f n [n(3ai + a 2 ) + 3(ai + a 2 + a 3 )] /i 2 

+n(ai + a 2 + a 3 )/i 3 (Bll) 
for the first sum. For the second sum, 

^2h,iJ a h,i,j (Xk x hXiXj) 

C 

= (n + /i)(2n + /i)(3n + + /i(n + u)(2n + /i)0 2 

+/i(n + /i) 2 3 + /i 2 (n + /i)0 4 + l^k 
= 6n 3 ai + n 2 (llai + 20 2 + /3 3 )/i 
+n(6ai + 30 2 + 20 3 + 4 )/i 2 

+(ai + 2 + 3 + 04 + 5 )/A (B12) 



52 a k,h,i( x k x hXi) = (xixxx^ai + (xix 2 x 2 )a 2 + (xix 2 x 3 )a 3 
( B9 )/m 

(B15) 



and 

^ ^ ^h,ijj (XkXfiXi 



(xixxxix^ai + (xix 2 x 2 x 2 )/3 2 



+ (XiXiX 2 X 2 )/3 3 + (XiXiX 2 X 3 )/3 4 . 

(B16) 

Thus s 4 = and we do not have the term (x\x 2 x%x^ . This 
changes the averages, (x\X\x 2 x 2 ) and (x \X\x 2 xz) as they 
were dependent on {x\x 2 x^x^ (see Eqs. (B6)). Eqs. (B7) 
do not change, but 5 = 0. Solving the two sums using 
these expressions and evaluating Eq. (All), specifically for 
n = 3, 



N(Ax* k el ) s 
5C 



3 3 
^ ^ ^k,h,i{XkXhXj) ^ ^ Q>h,i,j {XkXhXjXj) 
h,i h,i,j 

9 [4ai + 3a 2 - 2ft - ft] H 
+3 [6ai + 6a 2 + 3a 3 - 3/32 - 2ft - ft] /i 2 
+ [3 (ai + a 2 + a 3 ) - (ft + ft + ft + ft)] ^ 

(B17) 



Going back to our original Equation (All) and organising 
it in powers of a, we obtain, 



which can be written in the form of Eq. (B14). 

Finally for d = 3 and n — 2 the sums in Eq. (All) 
consist only of the following terms, 

2 

y^CLk,h,i(xkXhXj) = (xixix^ai + (xix 2 x 2 )a 2 . 

h,i 



(B18) 



N(Ax 8 k el } s 
5C 



^2ak,h,i(xkXhXi) - ^ a h ^j{xkX h XiXj) 

,i h,i,j 



and 



[n 2 (2a 1 (n - 1) + 3a 2 - 2f3 2 - #,)] /in 3 + 



[n ((3n — 3) ai + (n + 3) a 2 + 3^3 — 3/3 2 — 203 — 04 )] /i n' 



^ a h ^ j {x k x h x i x j ) = (xixixixi)ai + (xix 2 x 2 x 2 )0 2 

+ (xixix 2 x 2 )0 3 . (B19) 

The form of the aver ages does not change from the general 
2 3form given in Eqs. (B6) except for (x\X\x 2 x 2 ) which de- 



M fe 



pends on 54. Due to n = 2, 54 = and also 0^3 
04 = 0. For this special case thus, we have 



p 5 



+ A,[n ( ai +a 2 + a 3 ) - (Pi + p 2 + Ps + Pi + ft)] ^r? 



N(Ax s k ' 



sel\ 



sc 



(B13) 



Notice that the coefficients of the different orders of u con- 
sist only of the number of strategies and the payoff values. 



(2 2 
^ ^ ttk,h,i(XkXhXj) ^ ^ Q>h,i,j (XkXhXjXj^ 
h,i h,i,j 

= 4 [2ai + 3a 2 - 2ft - ft] p 
+2 [(3ai + 5a 2 - 3ft - 2ft)] p? 
+ [2 (a x + a 2 ) - (ft +ft + ft)]M 3 (B20) 
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which can be cast in the form of Eq. (B14) 



This case is actually very well studied. For multiple play- 
ers and two strategies it has been recently shown that the 
condition for strategy A replacing strategy B with a higher 
probability simply depends on the sums of the payoff values 
of the two strategies [121 [25]. This result is valid for ran- 
dom matching of players and small mutation rates. In our 
case, the condition for small mutation rates is obtained by 
checking the condition > 0, i.e., 



2ai + 3a 2 - 2 fa - fa > 0. 



(B21) 



Inserting the definitions of ai, a 2 , Pi and fa and rearrang- 
ing leads to 

2ai 5 i 5 i+ai 5 i ?2 + ai,2,i + 2ai, 2 ,2 

> 2a 2 ,i,i + a 2 ,i,2 + «2,2,i + 2a 2 ,2,2 (B22) 

Under random matching we have &i,i, 2 — &i,2,i and 
a 2,i,2 = ^2,2,i- Thus, this is equivalent to 

«1,1,1 + fll,l,2 + ^1,2,2 > «2,1,1 + 02,1,2 + a2,2,2(B23) 



which is the condition derived in [121 125] . When we do the 
same analysis for n = 2 and increasing d, and compare the 
Lk for each d, we will find a general form of the condition 
already given in [12| |25] for small mutation rates. 



Appendix C: Calculating probabilities based on Coalescence 
Theory 

In the coalescence approach, we take a sample from the 
present generation and look back in time with respect to the 
sample. Consider two copies of a gene. Sometime back in 
the past they come together to a common ancestor. This 
means the lineages of the two copies "coalesce" back in 
time. In general if we have a sample of d individuals from 
the present then sometime back the lineages of two of the 
individuals will coalesce and there will be d — 1 individuals. 
In all thus there will be d — 1 coalescence events until we 
arrive at the most recent common ancestor, the root of the 
coalescent. [2TH24] showed that the mathematical process 
of joining lineages leading up to the common ancestor can 
be analytically understood. He also showed that the coa- 
lescent encompassess a broad class of population dynamics 
models including Wright-Fisher and Moran processes. 

There are three assumptions of the most basic coalescent 
theory [56], 

• The population is not subdivided or structured. 

• The population size remains constant over time. 

• Genetic differences have no effect on the fitness of 
an individual. In our case this means that different 
strategies have the same fitness, the neutral case. 

We follow the approach developed in [2] [3] . In a neutral 
Moran process two individuals will have the same ancestor in 
one update step with probability 2/N 2 . We use a continu- 
ous time limit by rescaling the time such that r = t(2/N 2 ). 



We determine the results for a large, but finite population 
size N. 

The beauty of the coalescence process lies in the separa- 
tion of the genealogical part and the mutation process. This 
is due to the assumption of neutrality. Mutations occur at 
the rate of ji/2 where /i = Nu and u is the probability with 
which the offspring obtains any one of the n strategies at 
random. The mutation probability u can range from to 1, 
but when the mutation probability is 1 then the strategies 
would oscillate. Hence we rescale the mutation rate by 1/2. 
It has been shown by [22] [23] that when N is large, the 
coalescent time is exponentially distributed as, 



/ i(r ) = [!VG> 



(Cl) 



On each trajectory no mutation occurs in time r with prob- 
ability 



7 = e 2 . 



1. Calculation of S2 



(C2) 



First we repeat the derivations of [3] for completeness 
of the process. Also this will help simplify the terminolo- 
gies used in the next subsection. The quantity s 2 is the 
probability that two individuals chosen randomly in a neu- 
tral coalescent process have the same strategy. According 
to the coalescent back in time there was a single common 
ancestor of the two chosen individuals. Immediately after 
the ancestor split there were two individuals of the same 
type. Thus the ^ family consists of only one configura- 
tion, a pair of identical individuals. From then onwards to 
r 2 , mutations can play a role. Hence, the probability that 
the two individuals drawn have identical strategies when at 
the s 2 family level they have identical strategies, is given by 

*[2] 



*[2] 



(r)= 7 2 + ^ 7 (l- 7 ) + i(l- 7 ) 2 . (C3) 



The index [2] in the superscript describes the composition 
of the configuration. In this case denoting that both the 
individuals are of the same strategy. The terms on the right 
hand side from first to last can be described as follows, (i) 
None of the trajectories mutate and hence the individuals 
have identical strategies with probability 1. (ii) At least one 
mutation occurs on one of the trajectories and the chance 
that the new strategy is identical to the other is 1/n. As 
there are two trajectories this can happen in two ways, (iii) 
When both the trajectories mutate the first one gets some 
strategy with probability 1 and the second also mutates to 
the same strategy with probability, 1/n. 

This has to be weighted by the probability that we begin 
with two identical individuals at the s 2 family level. As 
this is the only possible configuration, the probability is 1. 
Further we also need to integrate with the coalescent time 
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density (Eq. ( |Cip ) to finally get S2 as, 

POO 

8 2 = 1 sf\r)f 2 {T)dT 



n + ii 
n(l + 



(C4) 



This is a the case of a fc-allele Moran model with replace- 
ment [7]. 



2. Calculation of s 3 

Now we take a step further. What is the probability that 
three randomly chosen individuals will have the same strat- 
egy ? The distribution of coalescent times is given by the 
density function for the coalescent event for three individu- 
als which is given by fs(r) = 3e _3r . 

Similarly as above we first investigate the s% family. At 
time T2 in the coalescent tree, one of the two individuals 
splits. Thus in the s% family, two individuals will always 
have identical strategies. In all there can be only two con- 
figurations, all three are identical or two have the same 
strategy and the third differs. 

We consider the two cases separately. If all three individ- 
uals have the same strategy at the S3 family level, then the 
probability that they have identical strategies after time r 
is, 



*[3] 



(t) = 7 3 + -7 2 (1 - 7) + t7(1 - 7) 2 + t(1 



■7) 3 - 

a a- ib~ 

(C5) 

If two individuals have the same strategy and the third one 
is different at the S3 family level, then the probability that 
they have identical strategies after time r is given by, 



*[2|1] 



(r) 



7 3 

1 , 



1 



-7 



2 (1 



7) 



77(1 



-i) A 

7 -(l-7) + ^7(l-7)" + ^(l-7) a - (C6) 



1? 
1 



1 



(1 



In the superscript the index [2|1] denotes that two indi- 
viduals are of the same strategy and one is of a different 
strategy. In this case we see that the first term for all three 
trajectories not mutating vanishes. This is because when 
we begin with the case when all the individuals do not have 
identical strategies, they cannot be identical later in time if 
no mutation occurs. 

To get the full probability 83 we need to weight the above 
two cases with the probabilities of their realizations. Three 
individuals will be the same at the s% family level if the two 
individuals at T2 are identical. This happens with probability 
52. The probability that they are not the same is thus I — S2. 
Putting in these weights and integrating over all possible 
times, we get 53 as 



S3 



rOO 

= s 2 S * [31 (r)/ 3 (r)dr 
Jo 

_ (n + fi)(2n + fi) 
~ n 2 (l + ^)(2 + /x)' 



(1-S2 



Jo 



*[2|1] 



8 2 family 




FIG. 4. The coalescent as it evolves through time. The 
probability that at time n the i individuals have the same 
strategy is given by Si. Immediately after n there are i + 1 
individuals. The strategy configuration at that time point 
depends if Si was 1 or not. If not then exactly what was 
the configuration? All these factors determine the possible 
configuration of the immediate i + 1 individuals and these 
different possibilities are grouped in the s* +1 family. 



3. Calculation of sa 

Here we calculate S4, i.e. the probability that four ran- 
domly chosen individuals have the same strategy out of a 
collection of n strategies. 

We are interested in the probability that the four leaves of 
the coalescent have the same strategy, cf. Fig. [4] At time 
T3, two of the four trajectories coalesce with rate 1. Hence 
there is a coalescence at rate 6, and the density function 
is given as, f^{r^) = 6e _6r4 . Before the bifurcation occurs 
at T3 the three players can have the same strategy with 
probability 83 or at least one is different with probability 
I-53. 

If the three players have the same strategy then immedi- 
ately after the coalescence there will be four players with the 
same strategy. If the three players do not have the same 
strategy then there are three different possible configura- 
tions. This is the family of configurations we denote by s\. 
Thus beginning with four individuals of different configura- 
tions we are interested in the probability that after time r 
all four of them will have the same strategy. 

The s% family consists of four cases: 



{r)h{r)dT 
(C7) 



Four identical individuals (Fig. [5] , s*^). In this 
case they will be the same at time T4 if none of them 
mutate. If one of them mutates that can happen 
with probability 4(7) 3 (1 — 7) and they are the same 
with probability 1/n. Similarly, we can write down 
when two or three or all can mutate and we get the 
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*[3|1] 



.*[2|2] 



,*[2|1|1] 



FIG. 5. The si family. All possible starting configurations 
where there are 4 individuals. Two of them have the same 
strategy. The figure shows all the possible combinations for 
the remaining two individuals. 



we need to weight each of the scenarios with the probabil- 
ity of the realization of the starting configuration. E.g. if 
the system reaches the state of all individuals having the 
same strategy from the second element of the si family i.e. 



*[3|1] 



then we have to weight it by the probability of that 



configuration, three of the same type and one different, Fig 
[5] This is possible if at T3 we do not have all three of the 
same type, but they must be of one of the the type (X1X2X2} 
or (X2X1X2) or (X2X2X1). Not only this, but the bifurcation 
should occur at one of the two identical types (#2) and not 
the different type (xi), the probability of which is |. Thus 

we have to weight S4' 3 ' 1 ' by | x (s 2 — S3) x 3. We calculate 
these weights for all the family members of si and thus get 
an expression for 84 as, 



expression, 

s f] (r)=7 4 + l 7 3 (1 _ 7) + _l 7 2 (1 _ 7)2 

n n z 

+A 7( i_ 7) 3 + J_ (1 )4 , 

n 6 n 6 



(C8) 



• Three of a kind (Fig. [5] , s^ 3 ' 1 ^). If only three are 
the same then if no one mutates it is impossible for all 
four to be the same at time T4. Similarly we can argue 
what happens if one, two, three or all four mutate and 
we get the expression for s^ 3 ' 1 ^: 



54 = 53/ sl(r)f A {r)dT + 2(s 2 - s 3 ) / s 
h Jo 

+(s 2 -s 3 ) sf l2 \r)U(r)dT 
Jo 

nOO 

+(1-3 S2 + 2 S3 ) / S I [2|1|11 (r)/4(r)d7 
Jo 



4 [3|11 (r)/ 4 (r) C Zr 



(3n + /i)(2n + fj,)(n + /x) 
n 3 (l + /i)(2 + /i)(3 + /i) ' 



(C12) 



4. Calculation of S4 



*[3|1] 



(r) = _ 7 3 (1 _ 7) + _ 7 2 (1 _ 7) 2 

+ ^ 7 (l-7) 3 + i(l-7) 4 - (C9) 



• Two pairs (Fig. [5] , s^ 2 ' 2 ^). At least two need to 
mutate such that we can end up with four identical 
individuals. Additionally the two mutating must be- 
long to the same pair. The last two terms are the 
same as before: 



*[2|2] 



(r) 



4t 2 (1-7) 2 + 4^(1 "7) 3 + ^(l -7) 4 - 

n l n 6 n 6 

(CIO) 



• Single pair (Fig. [5], S4' 2 ' 1 ' 1 '). At least two mutations 
are necessary for all four individuals to have the same 
strategy. The two mutations have to be on the tra- 
jectory of the non-paired individuals. Again the last 
two terms are the same as before: 



*[2|1|1] 



(r) 



r7 2 (l- 7 ) 2 + 7 (i_ 7 )3 + —(l- 7 ) 



n° 



(Cll) 



To obtain the final probability S4 (all four individuals have 
the same strategy), we combine all the above scenarios. But 



Here we calculate the probability 54 of picking four indi- 
viduals in the stationary state all having different strategies. 
As before we can have four different starting configurations, 
the same as shown in Figure [5] 

Hence basically now we want to calculate the probability 
that starting with each of the s| family members what is 
the probability of ending with all different individuals: 



• Four identical individuals (Fig. [5], S4' 4 '). We term 
the probability to start with four identical strategy 
individual to four different strategy individuals to be 



For four to be different at least three have to 



mutate. It can be calculated as follows, 



*[4] 



(r) = 4 7 (1 - 7 ) : 



,(n- l)(n-2)(n-3) 



+ (l- 7 ) 4 (n ~ 1)(n ~, 2)(n ~ 3) . (C13) 



• Three of a kind (Fig. [5], s^ 3 ' 1 ^). For all four individ- 
uals to be different now we need at least two individ- 
uals to mutate as we already have one individuals of 
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a different type. Hence, 



density and add them all together. Hence, 



*[3|1] 



( T )= 3 7 2 (l-7) : 



,(n - 2)(n - 3) 



+47(1-7) 



3 (n-l)(n-2)(n-3) 



+(1-7) 



4 (n-l)(n-2)(n-3) 



(C14) 



• Two pairs (Fig. [5], s^ 2 ' 2 ^). Here again we need at 
least two individuals to mutate for all the individuals 
to be different. Of the two individuals mutating each 
should be of different types. Hence, in all there are 4 
such combinations. 



^ [2|2] (r)= 4 7 2 (l- 7 y 



,(n-2)(n-3) 
ri 2 

3 (n-l)(n-2)(n-3) 



+4 7 (1 - 7) 

lb" 

(n-l)(n-2)(n-3) 
+0--V Zh • 



(C15) 



Single pair (Fig. [5], s^ 2 ' 1 ' 1 ^). For this starting con- 
figuration a single mutation is enough to create all 
different individuals provided it happens in one of the 
paired individuals. If two individuals are to mutate, 
then except for the two unpaired individuals together, 
all other groupings of two can give four different in- 
dividuals, hence in 5 different ways, 



s 4 = s 3 / sl(r)f4(r)dr + 2(s 2 - S3) 
Jo 

POO 

+(s 2 -s 3 ) sf l2 \r)U(r)dT 



*[3|1] 



(r)h(r)dT 



+ (1_ 3S2+2S3 ) / sf W \ T )U(T)dT 
JO 

/x 3 (n-l)(n-2)(n-3) 
n 3 (l + fi)(2 + fi){3 + n)' 



(C17) 



Due to the notational challenge, possible errors can arise 
hence to check our results, we simulated a neutral Moran 
process and computed the different averages, (xiXiXiXi), 
(#1X1X2X2), (#i#i#2#3) and (X1X2X3X4). 
These quantities depend on all the probabilities calculated 
in the Appendix namely S2, S3, £4 and S4. The results of 
the simulation and analytical method are shown in Figure 
□ 



10° 



10-- 



(x\X\XiX\) 




0.01 



0.05 0.1 

Mutation probability U 



_*[2|1|1], , 

V 0") 



2 7 3 (l- 7 ) 



(n-3) 



2 



+4 7 (1 
+(1-7) 



7) 
4 (n 



3O 



5 7 2 (l- 7 ) 
l)(ra-2)(ra-3) 



2)(n- 



l)(n 



-2)(n- 



3) 



^) FIG. 6. For a neutral Moran process with four strategies if we 
pick four individuals from the stationary state then the prob- 
ability that all of them have the same strategy is given by, S4, 
Eq. ( |C12| ). For four strategies (n = 4), the probability that 
all four have strategy 1 is S4/4 given by (x \X\X\X\). Similarly 



(C16)the probabilities for (X1X2X2X2), (X1X1X2X2), (x\X\X2Xz) and 
lb ~ (X1X2X3X4} are plotted as a function of the mutation prob- 

ability for a population size of N = 40. The symbols are 
To get the final probability s 4 we need to integrate all the simulations while the lines are the analytical results, 
different starting configurations over the coalescent time 
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